Monitoring Hadoop by Gurmukh Singh

Author:Gurmukh Singh [Singh, Gurmukh]
Language: eng
Format: azw3
Publisher: Packt Publishing
Published: 2015-04-28T04:00:00+00:00

Hadoop log level

In Hadoop, we can control the verbosity of the information logged into the logs in a similar manner to how we do with the syslog log level. In Hadoop, the log level is defined by the parameter HADOOP_ROOT_LOGGER in hadoop-env.sh.

The default configuration for this looks like the one shown as follows:

export HADOOP_ROOT_LOGGER="INFO,CONSOLE"

The preceding configuration is specified in the file hadoop-env.sh, and it states that the logging level is INFO and the destination is tied to the console.

This configuration could create a lot of noise as it is logging a lot of information. This can be modified by using the DRFA (Daily Rolling File Appender) and by reducing the log level, shown as follows:

export HADOOP_ROOT_LOGGER="WARN,DRFA"

Download

Copyright Disclaimer:
This site does not store any files on its server. We only index and link to content provided by other sites. Please contact the content providers to delete copyright contents if any and email us, we'll remove relevant links or contents immediately.